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ABSTRACT 

This study gathered information on the communication 
skills of preservice teachers using a rating scale based on the 
Speech Communication Association's (SCA) newly developed list of 
Teacher Communication Competencies. The research focused on two 
questions: (1) Can raters from different backgrounds with different 
levels of training reliably use a rating scale based on the SCA 
skills? and (2) Which items on the scale discriminate most 
effectively among preservice teachers? Four raters, including a 
Director of Student Teaching, a Professional Supervisor, a 
communication education specialist, and a speech communication 
professor, reviewed 10 student teacher video-taped lessons. Analysis 
of the rater's evaluations indicated that the rating scale had 
potential for an acceptable level of inter-rater reliability, and had 
some power to discriminate between effective and ineffective 
commuicaticn behavior in student teachers. Suggestions are made for 
improving the efficiency of rating scales in team evaluation of 
student teachar communication competencies. Copies of the rating 
scales are included in the report. (JD) 
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TEAM EVALUATION OF STUDENT TEACHER 
COMMUNICATION COMPETENCIES THROUGH VIDEOTAPE REVIEW 



Communication is essential to teaching. This fact, while certainly not a new idea, is 
receiving renewed attention. AACTE's Educating a Profession: Profile o f a Beginning Teacher 
(1983) asserts, ''A teacher's ability to communicate effectively is essentiar p. 50. Nutter 
stated (as cited in McCaleb, 1987) in A Case for Extended Programs of Initial Teacher 
Preparation, ^^communication skills are centra! to relating not only to students but also to 
professional collegues, parents, and other community representatives... It requires listening 
and reading skills as well as speaking and writing facility" (pp. 19-20). Recently, various 
"calls for" a focus on the oral communication competencies of teachers have been issued (e. g., 
Forster and Sloan, 1979: National Council for Accreditation of Teacher Education, 1979; Rubin 
and Feezel, 1985). These sources, and more, all point to the fact that the ability to listen, read, 
write, and speak effectively is esser»tial for effective teaching. 

Of these four skills, the two related to oral communication (speaking and listening) have 
received the least attention over the years in teacher training. Skills in writing and reading 
seem to be more easily identified and assessed. Oral communication skilb are more complicated 
and seem to involve a wider range of variables. In addition, a lack of agreement as to the 
constitution of the speaking aid listening skills necessary for effective teaching has hampered 
development of curriculum objectives and assessment procedures in teacher education programs 
(Cooper, 1987). 

This too, is changing. The Speech Communication Association recently commissioned a 
committee whose purpose was to review, synthesize, and summarize the oral communication 
competencies that would support effecii/e teaching in all content areas. These competencies 



were developed and defined in consuitatlon with professional organizations in teacher education 
and with a wide range of practicing teachers. These competencies are the basis for the study 
undertaken In this research. 

As attention to the definition of speaking and listening competencies was Increasing, so 
too was attention to the assessment of these skills. More and more states are beginning to 
include some type of oral communication assessment in teacher education programs. McCaleb 
(1987) found that more than half of ths 50 states have implemented assessments that include 
some measure of communication ability. Phi Delta Kappa reported in their newsletter (Joekel, 
1986, p. 1), "Competency assessment of teachers or teachers in training is occuring in 46 
states, only one state has no plans to do so." McCaleb found that at least 10 states are developing 
or giving serious consideration to adopting pf>rformance assessment. 

Given this evidence (and other evidence available) a conclusion can be drawn that 
communication skills are important io effective teaching and that more and more educators are 
giving serious attention to developing and assessing the communication skills of both in-service 
and pre-service teachers. However, as McCaleb (1987) points out, the meaning of 
"communication" differs considerably from state to state. It is possible for a teacher to be found 
competent in communication in one state and incompetent in another due to differences in both 
definitions of communication and methods of assessing those skills. In an effort to begin to bnng 
some coherence to the effort to define and assess the communication skills of teachers, the 
Speech Communication Association developed the Teacher Communication Competencies refered 
to above. The research project reported in this paper is based on the communication skills 
developed by the Speech Communication Association. 

The idea for this research project resulted from a conversation between the Director of 
Student Teaching and a speech communication faculty member on the Central Washington 
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University campus concerning the communication sl^ills of the University's student teachers. 
The two faculty members decided to pool their expertise and collaborate on a study that would 
indicate the communication sl^ills of student teachers as compared to the competencies ouiiined 
bySCA. 

This research then focused on two research questions: 

1. Can raters from different backgrounds with different levels of training reliably use a rating 
scale based on the SCA sl^ills? 

2. Which itemt, on the scale discriminate most effectively among pre-service teachers? 
METHODS 

To gather Information on these questions, a rating scale was developed based on the skills 
outlined by the Speech Communication Association (see appendix 1). The scale includes a line to 
indicate no rating (teacher had no opportunity to demonstrate the sl^ills during the segment 
viewed), a four point scale for rating the sidll, and a line for the confidence level of the 
judgment. 

From a series of student teacher video taped lessons there were 10 video tapes selecteJ 
randomly for tKs study. Those selected included 4 elementary student teachers, 5 secondary 
student teachers, and one special education teacher. These student teacher video tapes were 
rated by four raters including the Director of Student Teaching, a Professional Supervisor, a 
communication education specialist, and a speech communication professor. The raters were 
given a brief session to familiarize each with the rating scale. Then the videotapes of the student 
teachers were reviewed and rated by the raters. The videotape segments ranged in length from 
10 minutes to 20 minutes. 
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DATA ANALYSIS 

Ratings were analyzed using Pearson's T correlation coefficient. Coefficients were 
calculated for each possible pair of raters and for the raters as a group for each item on the 
ocale. The level of signfcance was set at .05 to determine which scale itevms received a 
signficant level of inter-rater reliability. 

RESULTS 

Our first research question was concerned with the ability of raters from different 
backgrounds to jse a scale based on the SZA teacher communication competencies reliably. Our 
research gave us a qualified "yes" to this question. 

Of the thirty-eight items in the original scale, twenty-three items appeared with at 
least one pair of raters showing significant correlation fp>.05). Of these twenty-three items, 
thirteen appeared with one pair of significant correlations, three items appeared witi". two pairs 
of significant correlations, six items appeared with three pairs of signficant correlations, and 
one item appeared with four pairs of significant correlations (out of a possible six). In addtion, 
a non-anchored single rater reliability coefficient of .4268 was caiucated across all four raters 
for all thirty-eight items. This reliability coefficient, non-anchored single rater reliability, 
is a useful indicator of the predictive reliability of the instrument. Essentially, if a single 
rater trained in using the instrument were to rate a teacher using this scale, this reliability 
coefficient would give us a measure of prediction on the ability of that rater to use the scale 
reliably. 

Of the fifteen items not showing a significant level of reliability, two were rated by the 
raters, but the ratings showed no signficant reliability. The thirteen other items appeared too 
infrequently in the videotape samples; insutHcient information was obtained to make any 



inferences about the rater reliability level. 

Our second research question concerned itself with the ability of the scale to 
discriminate effectively among the communication skill levels of pre-service teachers. Our 
primary tool for answering this question was a comparision between the scores of two of the 
raters. One rater was the Director of Student Teaching. This rater knew each of the subjects 
well, having supervised and advised them in their teacher training programs. His knowledge of 
the subjects gave him more information to use in making his ratings of the subjects. The other 
rater was a specialist in speech communication education. He did not know any of the subjects, 
and was seeing each for the first time. Our estimation of the discriminatory power of the scale 
was the level of agreement between these two raters. Results showed that nhirteen items showed 
a significant level of agreement between these two raters. 

CONCLUSK>B 

The purpose of this study was to gather information on the communication skills of pre- 
service teachers using a rating s^le based on the Speech C jmmunication Association's nawly 
developed list of Teacher Communication Competencies. This study gave preliminary answers to 
the two research questions developed tor this research. These answers, and other conclusions, 
are described below. 

1 . A rating scale based on the SCA competencies does have potential for an acceptable level of 
inter-rater reliability. The researchers suggest that f;j:!iher studies devote more time to rater 
training and practice. With greater familiarization, reliability estimates could rise 
significantly. 

2. The scale itself needs refinement in three areas. A) greater clarity in the wording to ensure 
a low level of item ambiguity, B) a!teration or potential deletion of the items that couldn't be 



rated due to non-appearance in the sample, and C) addition of items in \ho affective area to 
further discrinfnat.^ behavior in that important area. Work in these three areas could 
signficantly improve scaie validity. 

3. The researcher suggest that raters continue to rate video^tape sanpies. Using this method 
removes some of the anxiety a student teacher might feel if a rater was present in the room. The 
student would not need to l^now why the video-tape was being made. 

4. The scale does have some power to discriminate between effective and ineffective 
communication behavior in student teachers. Thi£ was demonstrated by the level of agreement 
reached by two of the raters who cx)me from very different backgrounds and who possessed very 
different levels of knowledge about the student teachers themselves. The discnminatory power 
of the scale can be further refined by additional development in the reliability and validity of 
the scale. 

The communication skills of teachers are important. The Speech Communication 
Association has made an effort to bring together some of the disparate conceptions of 
communication used in schools of education. This effort Is a much-needed step in bringing 
consistency to an area of study that has seen little consistency in recent years. If. as this study 
suggests, an assessment procedure can be developed that is based on the SCA competencies, then 
another useful step will have been taken. Teacher training programs may have, at some time in 
the near future, a clear set of communication competencies for teachers and a rating scale that 
effectively assesses those competencies. 

To move in that direction, a number of further research projects based on theses 
competences could be developed. For example, a control group of student teachers could be 
compared to an experimental group of student teachers that have received some instruction in 
classroom communication skills that are based on the SCA competencies. Other studies could 



Involve a) comparing grade and/or subject levels for similarities or differences in the use of 
these communication competencies; b) comparing the communication skills of student teachers 
to experienced teachers; and c) examining the communication skills of teacher that have been 
Identified as being known In their districts or schools as effective communicators. 

The authors suggest that there be more collaboration between teacher educators and 
speech communication educators to facilitate better communic:it;cn skills in our teachers of the 
future. 
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TEACHER COMMUNICATION RATING SCALE 



This version of the rating scale shows only those items that had at least one pair of raters with 
signficant level of reliability* All other items were deleted. In all cases except numbers 11 
asnd 17, items were deleted due to insufficient information from the videotape sample. Only 
numbers 11 and 17 were deleted due to Insignificant reliability. 

*** Indicates items with significant levels of reliability between four 
pairs of raters. 

Indicates items with significant levels of reliability between 
pairs of raters. 

* Indicates Items with signfbant levels of reliability between two 
pairs of raters. 

All other items listed had signficant levels of reliability between 
one pair of raters. 
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TEACHEFTS COMMUNICATION RATING SCALE 



Raiings do not need to be made in order presented here. In some instances, the rating for a 
behavior may be done during the observation. In other instances, the rating may need to be done 
at the completion of the observation to in^^re an adequv-le sample, 
1. INFORMATIVE MESSAGES 
a. sending 

M. Structures informative messages effectively by using devicei> such as intitial 
partitions, transitions, internal 
summaries, and concluding summaries. 



1-NR ? 3 4 5 6-CJ 

'2. Amplifies information effectively through the use of verbal and audio-visual 
supporting materials. 



1-NR 2 3 4 5 6-CJ 

'3, Asks effective questions to assess student understanding of information given in 
lectures. 



1-NR 2 3 4 5 6-CJ 

•M, Presents information in an animated and interesting way. 



1-NR 2 3 4 5 6-CJ 

b. Receiving 

•1, Is able to identify main point of student comment. 



1-NR 2 3 4 5 6-CJ 

3, Can evaluate the adequacy of verbal supporting materials. 

Tnr' T 3" 4" 5" 6-CJ 
••4. Can formulate questions that probe for the informative content of messages. 



1-NR 2 3 4 5 6-CJ 

2. Affective messages. Teacher should demonstrate competence in sending and receiving 

affective messages {l.e„ messages thatexpress or respond to feelings), 
a, sending affective messages 
**1. Express positive and negative feelings about self to students. 



1-NR 2 3 4 5 6-CJ 

4, Demonstrate interpersonal openness, warmth, and positive regard for students. 



1-NR 2 3 4 5 6-CJ 

5, Demonstrates energy and enthusiasm when relating to students. 



1-NR 2 3 4 5 6-CJ 
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b. receiving affective messages. 

2. Invite students to express feelings. 



1-NR 2 3 4 5 6-CJ 

5. If necessary, offer advice tactfully. 



1-NR 2 3 4 5 6-CJ 

Imaginative messages. Teacher should demonstrate competence in sending and receiving 
• imaginative messages (I.e., messages that speculate, theorize, or include fantasy), 
a. sending 

1 Use vivid descriptive language. 



1-NR 2 3 4 5 6-CJ 

2. Uses expressive vocal and physical behavior when creating or recreating 
examples, stories, or messages 
from exemplars. 



1-NR 2 3 4 5 6-CJ 

b. Receiving 
**1 . Respond to imaginative messages enthusiastically. 



1-NR 2 3 4 5 6-CJ 

**2. Be non-directive when encouraging student creativity. 



1-NR 2 3 4 5 6-CJ 

Ritualistic messages. Teacher should demonstrale competence in sending and receiving 
ritualistic messages (i.e., messages that serve 1o maintian and facilitate social interaction), 
a. Sending. 

1. Demonstrate approprlato behavior In performing ev&ryday speech acts such as 
greeting, tum-taking, and ieave taking. 

1-NR 2 3 ~~4" 5~ 6-CJ 



2. Model appropriate social amenities in ordinary classroom interaction. 



1-NR 2 3 4 5 6-CJ 

3. Demonstrate competence when participating in or role-playing interviev/s, 
conversations, problems solving groups, legislative groups, and public 
ceremonies. 



1-NR 2 3 4 5 6-CJ 

b. Receiving. 

1. Recognize when students perfomi everday speech acts appropriately. 



1-NR 2 3 4 5 6-CJ 
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5. Persuasive messages. Teacher should demonstrate competence in sending and receiving 
persuasive messages (i.e., messages that seek to convince), 
a. sending 

3. Offers sound reasons and evidence in support of ideas. 



1-NR 2 3 4 5 6-CJ 

5. Demonstrates a preference for reason-giving over power moves when Interacting 
with students. 

1-NR 2 3 ~4 5^ ~6-CJ 

b. receiving 

3. Evaluates evidence and reasons presented. 



1-NR 2 3 4 5 6-CJ 



TEACHER COMMUNICATION RATING SCALE 



Teacher name Date 



School Evaluator 



Grade Level Student Teacher? 

Certified? 

This evaluation form follows the Speech Communication Association's newly develop description 
of teacher communication competencies. It can be used as a basis for observation in evaluating 
the classroom communication skills of student teachers to determine the presence or absence of 
these communication skills. 

It is suggested that the following criteria for rating each item be followed for consistancy in 
using this instrument. 

Let a #1 rating mean Behavior did not appear in this observation 

Let a #2 rating mean Opportunities foi behavior were present, but student did not 

demonstrate the behavior at the appropriate time. 

Let a #3 rating mean Behavior demonstrated occassionally 

Let a #4 rating mean Behavior demonstrated consistantly, effectiveness average. 

Let a #5 rating mean Behavior demonstrated consistantly with obvious skill. 

Column #6 - Confidence of judgment. In observing a teacher for a short period of time, 
not all behaviors may be judged with equal confidence. Column #6 asks the rater to give a 
confidence level for the judgment of the indicated skill. 

•Rating of 1 indicates low confidence in judgment. There were low levels of the 

observed behaviors. 
-Rating of 2 indicates average confidence in judgment. 

-Rating of 3 indicates high confidence in judgment. 
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TEACHEFTS COMMUNICATION RATING SCALE 

Ratings do not need to be made in order presented here. In some instances, the rating for a 
behavior may be done during the observation. In other instances, the rating may need to be done 
at the completion of the observation to insure an adequate sample. 
1. INFORMATIVE MESSAGES 
a. sending 

1. Structures informative messages effectively by using devices such as intitial 
partitions, transitions, internal 
summaries, and. concluding summaries. 



1-NR 



6-CJ 



2. Amplifies information effectively through the use of verbal and audio-visual 
G^.-Pporting materials. 



1-NR 



6-CJ 



3. Asks effective questions to assess student understanding ol information given !n 
lectures. 



1-NR 2 3 4 5 6-CJ 

4. Presonts information in an animated and interesting way. 



1-NR 



6-CJ 



b. Receiving 

1. Is able to identify main point of student comment 



1-NR 2 3 4 5 6-CJ 

2. Can identify structural patterns or problems of informative messages. 



1-NR 



6-CJ 



3. Can evaluate the adequacy of verbal supporting 
materials. 



1-NR 2 3 4 5 6-CJ 

4. Can formulate questions that probe for the informative content of messages. 



1-NR 



4 



6-CJ 



5. Can distinguish between messages which are delivered in an animated manner and 
those which are not. 



1-NR 



6-CJ 
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2. Affective messages. Teacher should demonstrate competence in sending and receiving 
affective messages (i.e., messages that express or respond to feelings), 
a. sending affective messages 

1 . Express positive and negative feelings about self to students. 



1-NR 2 3 4 5 6-CJ 

2. Express positive and negative feelings atx)ut students to students. 



1-NR 2 3 4 5 6-CJ 

3. Expresses opinions about classroom content, events, and real world occurances. 



1-NR 2 3 4 5 6-CJ 

4. Demonstrate interpersonal openness, warmth, and positive regard for students. 



1-NR 2 3 4 5 6-CJ 

5. Demonstrates energy and enthusiasm when relating to students. 



1-NR 2 3 4 5 6-NR 



b. receiving affective messages. 

1 . Recognize verbal and nonverbal cues concerning student feelings. 



1-NR 2 3 4 5 6-CJ 

2. Invite students to express feelings. 



1-NR 2 3 4 5 6-CJ 

3. Be non-judgmental in responding to student feelings 



1-NR 2 3 4 5 6-CJ 

4. Ask open-ended questions in response to student expressions of feelings. 



1-NR 2 3 4 5 6-CJ 

5. If necessary, offer advice tactfully. 

1-NR T 3" 4~ 5~ 6-CJ 
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3. Imaginative messages. Teacher should demonstrate competence in sending and receiving 
imaginative messages (i.e., messages that speculate, theorize, or include fantasy), 
a. sending 

1. Use vivid descriptive language. 



1-NR 



6-CJ 



2. Uses expressive vocal and physical behavior when creating or recreating 
examples, stories, or messages from exemplars. 



1-NR 



6-CJ 



b. Receiving 

1* Respond to imaginative messages enthusiastically. 



1-NR 2 3 4 5 6-CJ 

2. Be non-directive when encouraging student creativity. 



1-NR 



6-CJ 



4. Ritualistic messages. Teacher should demonstrate competence in sending and receiving 
ritualistic messages (I.e., messages that serve to maintian and facilitate social interaction), 
a. Sending. 

1. Demonstrate appropriate behavior in performing everyday speech acts such as 
greeting, turn-taking, and 
leave taking. 



1-NR 2 3 4 5 6-CJ 

2. Model appropriate social amenities in ordinary classroom interaction. 



1-NR 



6-CJ 



3. Demonstrate competence when participating in or role-playing interviews, 
conversations, problems solving groups, legislative groups, and public 
ceremonies. 



1-NR 



6-CJ 



b. Receiving. 

1. Recognize when students perform everday speech acts appropriately. 



1-NR 2 3 4 5 6-CJ 

2. Recognize appropriate and inappropriate performances of social amenities. 



1-NR 



6-CJ 
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3. Recognize competence and incompetence when students participate In interviews, 
conversations, problem-solving groups, legislative groups, and public 
ceremonies. 

1-NR 2 3 4 5" 6-CJ 

5, Persuasive messages. Teacher should demonstrate competence in sending and receiving 
persuasive messages (i.e., messages that seek to convince). 

a. sending 

1. Can differsniiate between fact and opinion. 

1-NR 2~ T" 4 5 6-CJ 

2. Can recognize audience factors which may encourage or constrain acceptance of 
ideas. 

TnpT 2" 3 4 5^ 6-CJ 

3. Offers sound reasons and evidence in support of ideas. 

1-NR 2" 3~ 4 ^ 6-CJ 

4. Recognizes underlying assumptions in one's own arguments. 

1-NR 2^ 3 4 5" 6-CJ 

5. Demonstrates a preference for reason-giving over power moves when interacting 
with students. 

1-NR 2^ 3 4 T" 6-CJ 

b. receiving 

1. Recognizes own bias in responding to ideas. 

1-NR 2~ 3 4 6-CJ 

2. Questions the adequacy of reasons and evidence given. 

1-NR 2~ 3 4 5^ 6-CJ 

3. Evaluates evidence and reasons presented. 

1-NR 2^ 3 4 6-CJ 

4. Recognizes underlying assumptions in arguments of others. 

1-NR 2 3 4 5^ 6-CJ 
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